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Abstract 


This  research  has  pursued  a  “proof  of  concept”  prototype,  named  the  CONOPS 
Navigator.  The  Navigator  is  intended  to  provide  a  3D  virtual  guide  through  the 
development  of  a  CONOPS,  and  also  to  integrate  various  tools  and  applications 
currently  in  use.  This  integration  is  a  widely-sought  capability,  one  which  will  enable 
current  CONOPS  developers  and  users  the  flexibility  to  import  and  export  analysis 
parameters  and  results  to  and  from  various  familiar  and  well-used  tools.  Legacy 
systems  are  a  fact  of  life  in  operational  concerns;  this  prototype  is  intended  to 
demonstrate  interconnectivity  on  a  limited  scale  between  specific  simulation  and 
mathematical  modeling  software  packages,  via  a  main  operational  environment.  This 
environment  was  built  using  a  game  development  environment. 

Although  originally  conceived  as  a  standalone  research  task,  the  potential  synergies 
from  merging  RT  31  with  RT  23,  and  then  combining  development  architecture  and 
strategies  with  RT  30,  became  apparent  once  development  had  begun.  Some  already- 
developed  external  interfaces  were  seen  as  adjuncts  to  the  activities  performed  in  RT  30 
and  these  interfaces  would  certainly  be  useful  in  the  future.  By  far,  the  greater  synergies 
in  the  development  effort  were  in  architecture  and  operational  issues  -  such  issues  are 
transparent  to  the  user  but  vital  to  successful  delivery.  Further  exploration  would  be 
toward  an  integrated  data-set  and  application. 

The  research  includes  approaches  to  implementing,  managing,  and  addressing  data 
impedance  challenges  between  applications  including  Excel,  @Risk,  and  MATLAB. 
OneSAF  was  investigated,  and  the  determination  is  that  further  training  in  OneSAF 
operation  is  required  in  order  to  interface  successfully  with  this  tool. 
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1  Summary 


The  Department  of  Defense  (DoD)  is  vigorously  pursuing  greater  efficiency  and 
productivity  in  defense  spending  so  that  it  can  continue  to  provide  the  armed  forces  with 
superior  capabilities  in  an  environment  of  flat  defense  budgets.  Toward  that  end,  the 
Office  of  the  Secretary  of  Defense  (OSD)  has  issued  new  acquisition  guidance  that  places 
increased  emphasis  on  system  engineering  early  in  the  lifecycle  to  balance  operational 
performance  with  affordability  and  has  established  the  System  Engineering  Research 
Center  (SERC)  to  create  the  tools  and  processes  needed  to  execute  this  guidance.  As  one 
of  its  research  areas,  the  SERC  has  put  forth  the  notion  of  a  concept  engineering  system 
for  agile  CONOPS  Development. 

Technical  Reports  SERC-2009-TR-003  and  SERC  2010-TR-007  provide  a  compelling 
vision,  a  feasibility  assessment,  and  an  initial  process  definition  for  Graphical  CONOPS 
development  environment  for  agile  systems  engineering.  Current  research  will  focus  on 
creating  an  initial  prototype  to  demonstrate  a  cohesive  and  easy  to  use  collaborative 
concept  engineering  system  applicable  within  the  DoD  acquisition  domain. 

Consistent  with  RDECOM’s  vision  and  mission  to  be  the  Army’s  primary  source  for 
integrated  research,  development  and  engineering  capabilities  to  empower,  unburden, 
and  protect  the  Warfighter,  this  research  topic  calls  for  the  creation  of  an  early 
prototype  of  the  envisioned  collaborative  concept  engineering  system  demonstrated 
using  RDECOM-ARDEC  modeling  and  simulation  infrastructure,  RDECOM-ARDEC 
generated  concepts,  and  RDECOM-ARDEC  generated  scenarios.  Further,  it  will 
exercise  all  three  stages  of  the  agile  CONOPS  development  process  through  the 
prototype  demonstration  and  assess  strengths  and  weaknesses  to  guide  improvements 
for  future  prototypes. 

This  research  pursued  the  a  proof  of  concept  prototype  dubbed  the  “CONOPS 
Navigator”.  This  prototype  provides  a  3D  virtual  guide  intended  to  assist  one  assigned  to 
CONOPS  development,  through  the  setup  of  individual  analysis  tools.  Further 
exploration  would  be  toward  an  integrated  set  of  data  modeling  tools,  able  to  seamlessly 
transfer  data  from  one  application  to  another  via  the  CONOPS  Navigator  main  lobby. 
This  task  focused  solely  on  the  initial  stages  of  data  exchange  and  manipulations 
between  various  standalone  applications,  including  Excel,  @Risk,  and  MATLAB. 

OneSAF  was  evaluated  but  not  implemented  in  this  task. 
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2  Introduction 


It  is  believed  that  the  3D  gaming  technologies  available  today  can  be  used  to  provide  a 
useful  “front  end”  to  the  concept  engineering  process.  Selection  of  the  correct  game 
development  platform  will  be  critical  to  this  implementation. 


2.1  Use  of  Gaming  Technology  as  Conduit  for  Interoperability 
Communications 

In  early  2011,  under  RT  003,  gaming  technology  was  investigated  as  the  core  backbone 
link  between  all  the  CONOPS-specifics  functionality  -  including  scenario-building, 
simulation  using  various  third-party  vendor  packages,  and  generating  SysML/XML 
output  from  vendor  offerings  already  in  use  by  soldiers  in  the  field.  To  determine  which 
platform  to  select,  a  broad  range  of  available  gaming  environments  were  examined: 


Table  1:  Game  Development  Engines 


Torque  2D 

Unreal  DK 

Vicious 

Torque  3D 

ID  Tech  (Doom  3) 

Open  Simulator 

Quest  3D 

Cry  Engineer 

C4 

Unity 

MS-XNA 

Gamebryo 

Unity  Pro 

Adobe  Flash 

Dark  Basic 

Unreal  Engine 

Source 

Open  Simulator 

The  survey  examined  qualitative  evaluation  of  each  platform  on  a  number  of  criteria 
within  several  overall  categories,  as  shown  below: 

Features/Capabilities 

-  Multiplayer 

-  3D/2D  representations 

-  Specific  comparative  strengths  and  limitations 

-  Development  languages  and  physics  engines  supported 

Deployment 

-  Client-Server  capability 

-  Web,  PC,  Mac  supportable 

-  Minimum  CPU  and  RAM  required 

-  Video  card 

-  Minimum  bandwidth 
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Compatibility  with  Open  Source 

-  Source  code 

-  Open  source  components 

-  Open  interfaces 


Cost: 

-  per  seat 

-  to  deploy 

-  license  specifications 

The  evaluations  of  the  software  packages/environments  along  these  dimensions  are 
shown  in  the  figure  below. 


Of  all  the  criteria  evaluated,  several  dominated  the  decision-making  process;  most  of 
these  concerned  development  and  deployment.  These  included  (in  no  particular  order) : 

•  an  active  and  responsive  user  community, 

•  the  ability  to  port  to  different  platforms  easily, 

•  the  ability  to  easily  support  multiple  developers, 

•  providing  code  control  (though  this  is  not  a  production  environment), 

•  supporting  a  diversity  of  programming  languages  transparently,  and 
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•  the  ability  to  either  have  or  incorporate  open  source  components. 

In  today’s  environment  of  flat  defense  budgets,  cost  is  also  a  factor,  although  site-wide 
and  server  licenses  may  help  mitigate  concerns  that  per  seat  licenses  may  incur. 

Although  not  stated  as  one  of  the  “critical”  components  of  the  decision-making  process, 
the  availability  of  scalable  3D  models  was  also  crucial.  The  applications  will  be 
operating  in  (and  as)  a  visually-based  immersive  environment;  having  the  models  and 
simulation  as  realistic  as  possible  will  help  increase  the  probability  of  acceptance  and 
usage  by  the  eventual  field  users.  3D  models  can  also  have  a  considerable  cost  factor. 
For  this  task,  the  group  utilized  3D  models  that  were  found  at  no  cost,  although  the 
eventual  selected  platform  does  have  extensive  libraries  of  3D  models,  some  available  at 
no  cost  or  for  a  nominal  fee. 

Most  of  the  platforms  also  had  other  limitations,  another  factor  when  selecting  the 
platform  -  cost  and  point-of-view  being  two  major  considerations. 


2.2  Final  Platform  Selection 

As  can  be  seen  in  Figure  1,  many  of  the  investigated  platforms  have  major  drawbacks 
(shown  in  red).  Chief  among  these  was  their  inability  to  deploy  on  the  Web.  A 
secondary  consideration  for  this  phase  of  our  research  task  is  the  ability  of  the  tool  to 
interface  with  open  source  code  and  components. 

The  selected  platform  was  Unity  3D  Pro.  The  learning  curve  for  developers  was  found  to 
be  less  daunting  than  that  of  most  of  the  other  platforms,  being  more  intuitive  and  the 
facility  to  develop  and  deploy  components  was  relatively  easily-acquired. 

Unity  3D  Pro  has  an  asset  server  which  acts  as  a  central  code  storage  and  a  rudimentary 
code  control  mechanism.  It  has  a  rich  library  of  models,  environments,  scripts,  and 
other  development  components  available,  either  free  or  at  a  nominal  cost. 

Unity  3D  Pro  supports  a  number  of  programming  languages:  C#,  Boo  (Python),  and 
Javascript.  The  Unity  physics  engine  supports  movement,  collision,  gravity  for  solid 
objects,  and  users  can  modify  textures/meshes.  This  ability  will  be  critical  if  terrain 
generation  from  various  USGS  databases  is  to  be  evaluated. 

Unity  3D  Pro  has  a  large  user  community  which  is  extremely  responsive  to  posted 
questions,  and  a  forum  containing  posted  solutions  to  many  commonly-found  problems 
or  desired  effects.  As  this  research  task  was  focused  mainly  on  interfaces  between  3rd- 
party  software,  we  did  not  find  solutions  in  user  community  resources  for  these  tasks, 
however  the  resources  did  help  when  implementing  some  of  the  more  complex  model 
representations  and  movement. 


Contract  Number:  H98230-08-D-01 71  DOl ,  TT02,  RT0031 

Report  No.  SERC-2011-TR-031 
March  23,  2012 


UNCLASSIFIED 


3  Work  Performed 


Kickoff 

1/24/2011 

In  Progress  Review  (IPR)  l 

5/12/2011 

In  Progress  Review  (IPR)  2 

9/12/2011 

In  Progress  Review  (IPR)  3 

11/28/2011 

Final  Review/Presentation 

3/23/2012 

This  work  was  performed  in  two  stages: 

-  The  first  stage  was  the  development  of  standalone  applications,  to  validate  the 
conceptual  and  architectural  approach  of  each  interface  separately. 

-  The  second  was  to  merge  the  various  standalones  into  one  umbrella  executable. 


bdd  [Package]  ICES  Domain  [ICES  Domain  Diagram] 


Figure  2  ICES  Domain 
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This  is  a  proof-of-concept  research  task  -  that  implies  that  the  software  must  perform 
within  a  relatively  flexible  set  of  criteria;  it  is  not  a  production  system.  Of  necessity, 
major  error-handling  is  not  a  factor  in  our  evaluation  of  preparedness,  but  reasonable 
error-handling  and  performance  issues  are  addressed. 

Our  first  step  was  to  develop  the  interface  between  Unity  and  Excel  -  and  this  approach 
brought  us  an  unintended  benefit.  An  interface  for  Excel  would  also  be  usable  for 
Microsoft  Word,  therefore  creating  a  conduit  to  save  results  from  simulations  run  in 
third-party  software.  We  began  with  a  CONOPS  Lobby  -  a  virtual  room  where  a  use 
could  choose  among  several  options  for  their  particular  need. 

-  Microsoft  Excel 

-  Anylogic 

-  @Risk  Simulation  libraries 

-  OneSAF 

-  Sparx  (SysML  package) 

-  MATLAB  (via  the  Decision  Support  Center,  DSC) 
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The  opening  scene  is  shown  below: 


1 

MSEicH 

OuStf 

VtSx 

H 

3|« 

OK 

• 

w  1 

Figure  4  CONOPS  Lobby 

3.1  Excel- Interface  &  Operation 

Upon  the  selection  of  Excel,  the  following  right-  and  left-hand  side  menus  appear: 


Figure  5  Excel  Input 


The  software  allows  the  user  to  specify  a  data  file  for  input.  Once  entered,  as  shown 
below,  the  user  can  select  from  various  result  options.  Here,  we  show  the  output 
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resulting  from  the  selection  of  all  the  available  general  statistics  for  the  dataset  provided 
in  the  test  file: 


The  user  is  then  given  the  option  to  export  the  results  data  directly  to  a  file  which  can  be 
stored,  or  to  open  the  results  data  in  a  Microsoft  Word  document,  for  further  viewing  or 
possible  manipulation. 


Figure  7  Browser  for  storing  output  as  Microsoft  Word  Document 
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Figure  8  Exported  data  to  Microsoft  Word  document 


The  use  of  Excel  is  enabled  by  C#  scripts  within  Unity  3D  Pro,  and  uses  two  external 
programs  for  initiating  10  Pipes.  The  two  external  programs  reside  in  a  special  Deploy 
folder,  and  must  be  present  for  the  application  to  successfully  call  the  Microsoft  Excel 
functions,  as  well  as  writing  to  a  Microsoft  Word  document.  This  is  an  example  of  the 
synergy  of  this  development,  as  well  as  the  benefits  of  using  named  pipes.  A  named  pipe 
is  an  extension  of  the  pipe  concept  on  Unix-type  systems,  and  serves  as  the  inter-process 
communication  conduit  for  the  data  stream  input  and  output.  A  named  pipe  is  system- 
persistent  and  exists  beyond  the  life  of  the  process,  which  requires  that  it  be  deleted 
once  it  in  no  longer  needed.  Once  the  process  connects  to  the  named  pipes, 
communication  between  applications  is  possible. 


3.2  @Risk  Simulation 

The  selection  of  the  @Risk  Simulation  libraries  leads  to  similar  input  screens,  although 
they  are  tailored  for  individual  input  -  characteristics  of  the  distributions  which  serve  as 
input  to  the  libraries. 
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Figure  9@Risk  Simulation  -  Output  of  LogNormal  Distribution 
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Figure  10  @Risk  Simulation  -  Output  of  PERT  Distribution 


The  calls  to  the  @Risk  simulation  SDK  libraries  are  made  via  Javascript.  The  return 
values  are  text,  and  the  graphic  representation  is  also  formatted  as  a  stream  of  text. 
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3.3  Decision  Support  Center 

The  decision  support  application  is  partitioned  into  three  sections,  each  of  which 
highlights  a  separate  interface. 


3.3.1  Vehicle  Simulation 

Upon  selection  of  the  Decision  Support  Center  application,  Vehicle  Simulation,  the 
following  initialization  screen  is  displayed. 


Figure  11  Vehicle  Simulation  Initial  Screen,  MATLAB  initialization  being  performed  (JLTV 

shown) 


The  user  can  use  the  slider  bars  shown  in  the  above  figure,  to  vary  the  distance  of  the 
simulation,  the  speed  and  acceleration  of  the  vehicle.  The  application  retrieves  vehicle 
specifications  and  parameters  from  an  Excel  file.  In  this  file,  each  sheet  represents  the 
specifications  of  a  vehicle  -  the  file  can  be  extended  and  modified  as  necessary  for 
additional  vehicles  (see  Figure  12  below). 
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Vehicle  1 

Name:  Humvee 

Vehicle  Attributes 

Speed  Distribution 

RiskLognorm 

Speed  (mean) 

46 

Speed  (std  dev) 

12 

MPG 

20 

Personnel  Capacity 

2 

Max  Speed 

65 

Max  Acceleration 

CaDacitv  model  | 

Infernal  Variables 

Externa  /  Variables  1 

Personnel  Capacity 

21 

Required  Passenge 

53 

MPG 

20 

Distance 

125 

Cost  of  Fuel 

3.3 

Ca/cufations 

Trips  Required 

27 

Trips 

Total  Distance  to  Travel 

6750 

Miles 

Fuel  Use 

337.5 

Gallons 

Fuel  Cost 

1113.75 

Dollard 

Travel  Time  [averaael 

146.7391304 

hours 

Fuel  Economy  Model  | 

Internal  Variables 

External  Variables  1 

MPG 

20  Distance 

125 

Cost  of  Fuel 

3.3 

Ca/cu/atiens 

Fuel  Use 

6.25 

Gallons 

Fuel  Cost 

20.625 

Dollars 

ResDonse  Time  Simulation 

1 

Internal  Variables 

Externa/  Variab/es  1 

Distribution  Type 

RiskLognorm 

Distance 

125 

Speed  mean 

46 

Cost  of  Fuel 

3.3 

Speed  SD 

12 

MPG 

20 

Calculations 

Average  Speed 

60 

MPH 

Response  Time 

2.083333333 

Hours 

1  Fuel  Use 

6.25 

Gallons 

Fuel  Cost 

20.625 

Dollars 

Figure  12  Sample  Excel  Vehicle  Definition  File 


The  application  also  shows  an  initialization  of  MATLAB,  prior  to  running  the 
application.  If  MATLAB  is  not  installed,  the  user  will  not  be  able  to  run  the  simulation. 
The  initialized  application  is  shown  in  the  next  three  figures;  the  first  is  a  3rd-person 
view,  the  second  is  the  overhead  point  of  view  built  into  the  application,  and  the  third  is 
a  ist-person  “driver”  view  from  the  vehicle  interior. 
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Figure  13  Vehicle  Simulation  Initial  Screen,  MATLAB  verified  (MRAP  shown)  3rd  Party  POV 
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Figure  14  Vehicle  Simulation,  Overhead  POV  camera 
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Figure  15  Vehicle  Simulation  Driver  POV 


The  algorithm  to  model  ideal  one-dimensional  motion  of  a  vehicle  over  a  specified 
distance  assuming  a  maximum  velocity,  acceleration,  and  jerk  used  in  this  simulation  is 
based  loosely  on  the  work  of  Richard  D  Peters  (Peters).  This  algorithm  runs  iteratively 
calculating  the  parameters  to  model  the  vehicle  at  each  step  of  time  and  accounts  for  the 
four  possible  outcomes  of  motion: 

•  max  velocity  is  reached, 

•  max  acceleration  is  reached  but  not  max  velocity, 

•  neither  max  velocity  nor  max  acceleration  is  reached,  and 

•  max  acceleration  is  not  reached  but  max  velocity  is  reached. 

The  MATLAB  program  was  then  integrated  with  the  Unity  platform  to  show  a  real  time 
representation  of  this  data  in  a  visual  simulation.  Unity  creates  a  TCP/IP  listening 
server,  opens  MATLAB,  connects  as  a  client  to  the  Unity  application  on  the  specified 
port,  sends  a  request  for  data,  and  then  waits.  During  this  time,  the  user  on  the  Unity 
application  is  given  time  to  choose  a  vehicle,  distance,  max  velocity,  and  max 
acceleration.  Once  MATLAB  initializes,  the  user  is  then  given  the  option  to  run  the 
simulation.  As  the  simulation  button  is  pressed,  data  is  passed  through  the  TCP/IP 
connection  to  MATLAB  which  interprets  the  input  and  begins  running  the  simulation. 
On  each  iteration  MATLAB  first  checks  for  a  command  from  Unity,  then  calculates  the 
next  set  of  parameters,  and  sends  them  to  Unity  over  the  network  connection.  As  Unity 
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gets  the  data  packets,  it  converts  them  to  the  distance  velocity  and  acceleration 
arguments,  moves  the  vehicle  appropriately  on  the  next  frame  and  updates  the  display 
for  current  position,  velocity,  and  acceleration.  At  any  point  during  the  simulation,  the 
user  can  pause  the  simulation,  restart  the  simulation  with  the  same  or  different 
parameters,  or  cancel  the  simulation  and  exit  to  the  main  menu.  This  is  achieved  by 
sending  a  command  packet  to  MATLAB  over  the  established  TCP/IP  connection  and 
allowing  the  MATLAB  program  to  process  the  command  and  act  accordingly. 

The  current  basic  formulation  of  the  MATLAB  model  does  not  yield  overly  powerful 
results,  but  it  proves  the  concept  of  a  real  time  simulation  built  around  the 
computational  power  of  MATLAB  and  the  visual  properties  of  Unity. 

Future  simulations  could  include  more  powerful  formulations  and  one  investigation  can 
include  a  feedback  loop  from  Unity.  For  example,  a  more  complete  model  could  be 
created  for  the  vehicles,  including  properties  like  torque  and  mass.  A  3D  path  could  be 
created  in  Unity  for  the  vehicle  to  follow  and,  as  the  vehicle  moves  along  that  path,  data 
could  be  sent  to  MATLAB  concerning  the  pitch  and  yaw  of  the  vehicle,  which  would 
affect  its  velocity  and  acceleration  characteristics.  As  this  data  is  sent  to  MATLAB  in 
each  frame,  the  subsequent  calculation  would  be  sent  back  showing  new  displacement 
acceleration  and  velocity  in  each  direction  as  well  as  about  each  axis. 


3.3.2  Vehicle  Allocation 

Upon  selection  of  the  Decision  Support  Center  application,  Vehicle  Allocation,  the  user 
can  select  the  comparison  of  vehicles  for  various  parameters,  the  first  one  shown  below, 
is  vehicle  carrying  capacity  -  in  this  case,  between  a  Humvee  and  a  JLTV. 
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Compare  Vehicles  I  Capacity  *  Close 

Capacity  Vehicle  1:  »  Humvee 

Fuel  Efficiency  JLTV 

Response  Time  I  '  Ml  13 

MRAP 

Stryker 

Vehicle  2:  Humvee 


•JLTV 
Ml  13 
MRAP 
*  Stryker 

Passengers:  |4  | 

Distance:  |6  | 

Fuel  Cost:  [3.95|  ] 


Run 


Figure  16  Vehicle  Capacity  Input  Screen  -  Comparing  Humvee  and  JLTV 


Compare  Vehicles 
Capacity 


Fuel  Efficiency 


Response  Time 


Analysis  Results 

•  Close 

Vehicle  Name  Humvee 

JLTV 

Trips  Required  2 

1 

Travel  Distance  24 

12 

Fuel  Use  1.2 

0.4 

Fuel  Cost  4.74 

1.58 

Travel  Time  0.5217391 

0.2222222 

Figure  17  Vehicle  Capacity  Output 


In  order  to  run  the  vehicle  fuel  efficiency  calculations,  the  initial  screen  presented  is: 
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*  Close 


Compare  Vehicles 
Capacity 
Fuel  Efficiency 
Response  Time 


Fuel  Efficiency 
Vehicle  1:  Flumvee 


Vehicle  2: 


Distance: 
Fuel  Cost: 


JLTV 
Ml  13 
MRAP 
*  Stryker 
Humvee 
JLTV 
Ml  13 
»  MRAP 
Stryker 

18 


(17.50 


Run 


Figure  18  Vehicle  Fuel  Efficiency  Initial  Screen 


In  this  case,  the  vehicles  being  compared  are  a  Stryker  and  an  MRAP,  over  a  distance  of 
8  miles  and  with  a  fuel  cost  of  $  17.50/gallon.  The  output  from  this  simulation  is  shown 
below: 


Compare  Vehicles 

Analysis  Results  *  Close 

Capacity 

Vehicle  Name  Stryker  MRAP 

Fuel  Efficiency 

Fuel  Use  0.2857143  0.2352941 

Response  Time 

Fuel  Cost  5  4.117647 

Figure  19  Vehicle  Fuel  Efficiency  Comparison  Output 
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3.3.3  Response  Time 

Upon  selection  of  the  Decision  Support  Center  application,  Response  Time,  the  user  can 
compare  the  fuel  usage  and  fuel  cost  between  two  vehicles  traveling  the  same  distance. 


*  Stryker 

Vehicle  2:  Humvee 

•  JLTV 
M113 
MRAP 
Stryker 

Distance:  |8 

Fuel  Cost:  1 5.50 

Run 


Figure  20  Response  Time  Input  Screen 


Compare  Vehicles  Analysis  Results  -Close 

Capacity  Vehicle  Name  Stryker  JLTV 

Fuel  Efficiency  Average  Speed  45.16708  53.99728 


Response  Time 

Response 

0.177120 

0.148155 

Time 

2 

6 

Fuel  Use 

0.2857143 

0.2666667 

Fuel  Cost 

1.571429 

1.466667 

Figure  21  Response  Time  Output  Screen 


The  calculations  for  the  vehicle  comparison  Capacity  and  Fuel  Efficiency  decision 
components  are  being  made  via  the  Excel  interface.  The  Response  Time  simulation  is 
handled  using  the  @Risk  Simulation  SDK  library  function. 
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4  Research/Questions  and  Lessons  Learned 


In  addition  to  RT  30,  RT31  evolved  from  the  initial  research  task,  RT3.  Since  RT31 
seeks  to  address  related  sponsor  defined  needs,  the  research  team  defined  a  high  level 
research  question  to  tie  together  the  RT  3/30/31  thread: 

Can  the  process  of  Concept  Engineering  improve  the  understanding  and  development  of 
a  concept  of  operations  using  gaming  technologies  along  with  an  interactive, 
collaborative,  and  graphical  environment? 

From  this  question,  each  research  task  contains  lower  level  research  questions  to 
address  specific  task  goals. 


4.1  Research  Questions  For  RT31 

•  Can  the  process  of  CONOPS  modeling  and  simulation  be  improved  through  the  use 
of  a  graphical  user  interface  which  would  serve  as  a  conduit  for  data? 

•  What  are  the  benefits  of  a  single  user  interface  for  the  tools  currently  in  use  for 
modeling  and  simulation  studies? 

•  What  are  the  drawbacks  of  a  single  user  interface  for  the  tools  currently  in  use  for 
modeling  and  simulation  studies? 

•  Does  real-time  collaboration  between  distributed  stakeholders  improve  the  CONOPS 
development  in  the  area  of  modeling  and  simulation? 

•  Can  a  real-time  collaboration  environment  enable  quicker  consensus  on  CONOPS 
generation? 

•  Are  there  new  or  specific  issues  in  asynchronous  software  development  in  an 
immersive  environment? 


4.2  Research  Lessons  Learned 

The  software  effort  is  actually  secondary  to  the  research  questions  being  posed,  yet 
consumes  what  seems  to  be  98%  of  effort  in  the  early  stages.  Several  categories  of 
“lessons  learned”  were  observed  during  this  effort: 
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4.2.1  Project  Management 

•  It  is  critical  to  continuously  monitor  migration  to  new  development  environment 
releases  -  we  now  only  migrate  as  a  team,  and  then  only  after  testing  current 
builds  in  new  release. 

•  Iterative  development  tasks  can  lead  to  redundant  efforts  and  conflicts;  a  lot  of 
time  and  effort  are  needed  to  avoid  wasted  effort  or  design  conflict. 

•  Use  of  Agile  processes  very  difficult  in  academia  -  neither  students  nor  faculty  is 
regular  in  their  schedules/ work  times.  Because  of  this,  the  use  of  Skype  and 
Google+  hangouts  can  be  effective,  especially  when  it  comes  to  review,  walk¬ 
through,  and  testing. 

•  Weekly  builds,  although  difficult  to  implement  at  the  beginning  of  the 
architecting  phase,  are  a  critical  factor  for  successful  completion.  The  reasons  are 
two-fold: 

o  They  also  build  team  cohesiveness,  and  focus  attention  on  research  and 
project  goals 

o  They  highlight  any  performance  or  operational  anomalies  -  code  that 
works  well  in  the  Unity  development  environment,  may  not  be  operational 
in  executable  build.  This  could  give  a  false  sense  of  security  to  developers, 
by  effectively  hiding  executable  conflicts. 

•  International  composition  of  workforce  has  its  own  challenges,  both  from  a 
language  side  and  a  cultural  side.  In  addition,  citizenship  requirement  for  some 
software  was  a  mitigating  factor. 

o  Clear  Skype  or  Google  +  hangouts,  or  even  written  word  communication 
can  be  challenging  and  clarity  can  suffer  when  there's  a  lack  of  visual  clues 

o  Video  conferencing  is  highly  preferred  over  voice-only  or  written 
communication. 

o  Face-to-face  remains  the  best  way  to  manage,  but  video  is  a  valuable  2nd 
best 

o  Avoid  idioms  when  describing  operational  desired  design  attributes  and 
functionality 

o  Analogies  can  work,  but  should  be  simple  and  clear 
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•  Measuring  progress  via  visible  functionality  is  not  helpful,  nor  is  using  long¬ 
standing  measures  such  as  SLOC.  Other  criteria  must  be  adopted  and  we 
propose  a  combination  of  SLOC  count  and  a  partitioning  of  categories  of  code: 
infrastructure,  actual  3D  object  presentation,  and  3D  model  manipulation 

o  Current  Statistics: 

■  SLOC  for  executable:  1270,  #  objects:  147 

■  SLOC  for  work  in  progress:  416,  #  objects:  63 

•  Organization  of  code  within  the  project  listing  is  critical,  especially  when  using  a 
multiple-developer  approach.  Naming  folders  clearly  and  grouping  scripts 
together  is  critical,  since  most  of  the  scripts  are  small  (and  again,  naming  clearly 
here  is  critical  to  efficient  searches). 

•  The  actual  3-D  models  used  are  free  for  academic  and  research  use  but  are  not 
free  for  commercial  distribution;  this  is  a  consideration  if  deploying  in  an 
organization,  so  factor  time  for  3-D  model  development. 

•  The  OneSAF  battle  simulation  package  is  highly  complex  and  requires  a  great 
deal  of  time  and  training  to  generate  output  capable  of  being  used  by  the  current 
application. 


4.2.2  Architecture 

•  Integration  of  previously-developed  standalones  into  an  overall  executable  build 
was  moderately  difficult.  After  consideration,  it  was  felt  that  clear  architecture 
and  preview  of  all  modular  activity  and  interfaces  would  be  a  more  practical 
approach. 

•  Early  integration  (and  builds)  would  also  assure  that  the  look-and-feel  of  various 
tool  interfaces  is  similar. 

•  All  architecture  and  data  objects  should  be  specified  as  completely  as  possible 
and  as  early  as  possible. 
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•  Because  architecture  (and  infrastructure)  is  not  “seen,”  the  work  done  is  not 
obvious  to  management/  customer  -  and  therefore  it  gives  the  impression  of  “no 
progress” 

•  Error  handling  architecture  should  be  context-reliant,  and  needs  to  be  addressed 
for  consistency 

•  Communication  issues  were  the  largest  stumbling  block  in  this  particular 
research  task.  The  interfaces  between  each  tool  and  the  Unity  3D  gaming 
platform  are  all  custom  built,  but  also  built  as  generically  as  possible,  to  support  a 
wide  range  of  calling  options. 

•  One  of  the  major  re-writes/re-working  of  architecture  centered  on  the 
performance  of  the  communication  interfaces  between  the  Unity  platform  and 
MATLAB  and  the  @Risk  libraries.  Although  this  is  “proof  of  concept”  software,  it 
was  felt  that  some  performance  considerations  needed  to  be  taken  into  account,  if 
the  software  is  to  be  successfully  deployed  on  a  large  scale  and  over  multiple 
platforms  with  multiple  users. 

4.2.3  Project  Code/Construction 

•  Asset  Server  Change  Control/Staging  Platform 

o  Individual  project  access  in  asset  server  needs  to  be  transparent  to  all 
developers 

o  Using  the  control  management  Asset  Server  was  more  complex  than 

anticipated,  and  there  was  no  easy  way  to  roll-back  to  a  previous  release  or 
version 

o  There  were  occasional  slow-downs  when  committing  to  or  updating  from 
the  Asset  Server 


-  Highly-modular  design  vies  with  programming  strategies  -  optimal  breakpoints 
must  be  developed 

o  Assignment  of  modular  design  elements  can  be  problematic  and,  because 
of  the  iterative  nature  of  design  and  development,  is  a  real  challenge 
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o  Realize  that  graphic  design  of  3D  models,  including  scaling  and 

manipulation,  took  longer  than  originally  anticipated;  this  is  not  due  to  the 
provenance  of  the  models,  but  is  inherent  to  3D  environments 


•  Avoid  manipulation  of  the  object  surface  meshes  -  in  order  to  indicate  a 
“selection”  insert  an  indicator  above  the  object  itself 

•  When  working  with  animation  and  interfaces  with  simulation  software, 
performance  and  communication  issues  may  take  additional  development  time. 

•  Actual  physical  movement  of  groups  (for  instance,  a  group  of  ground  vehicles,  or 
deployment  of  fleet)  will  be  crucial  as  the  design  moves  forward,  in  terms  of 
visual  representation,  as  well  as  generated  output 

•  Although  containment  of  objects  isn’t  a  large  consideration  in  this  RT  yet,  it 
should  be  kept  in  mind  as  an  issue  which  can  impact  performance,  storage,  and 
retrieval. 

•  Manipulating  colliders  is  how  objects  have  solidity  in  the  environment  -  this  is  an 
important  consideration  when  making  scenes  executable. 

•  Consider  the  use  of  multiple  cameras  as  a  default  mechanism  for  each  scene 
when  being  built,  and  provide  an  easy  toggle  for  users  to  switch  views. 

•  Terrain  generation  from  real-world  (USGS)  server  files  is  inefficient  at  this  time. 
The  size  and  complexity  of  the  USGS  library  files  were  too  large  to  scale 
effectively  in  the  task  time  frame.  In  addition,  the  problem  of  ground  cover  (tree 
canopy,  etc.)  is  an  issue  when  the  aim  of  the  simulation  is  to  depict  actual  ground 
condition.  The  team  was  able  to  successfully  import  small  ground  segments  via 
Google  Earth  and  Google  SketchUp  from  areas  which  has  little  or  no  ground 
cover. 
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5  Conclusions 


The  possibility  of  using  a  3D  gaming  environment  and  platform  as  a  window  into  the 
interfacing  and  use  of  various  simulation  packages  is  proven.  Three  third-party 
packages  (MATLAB,  Excel,  @Risk)  were  integrated  within  the  Unity  3D  gaming 
environment,  with  a  bookcase  paradigm  for  simulation  package  selection. 

Licenses  for  third  party  software  must  be  investigated  for  wide-spread  distribution  for 
this  software  -  the  development  group  has  used  academic  licenses  to  implement  the 
project  with  the  accompanying  software,  but  production  deployment  will  necessitate  a 
close  examination  of  distribution  and  use  of  independent  vendor  libraries  and  packages. 
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6  Recommendations  for  Continuation 


The  strategies  for  continuation  include  the  following: 

Future  work  would  encompass: 

-  the  investigation  of  interfaces  with  AnyLogic,  Sparx,  and  OneSAF 

-  terrain-mapping  capability 

With  regards  to  deployment,  future  work  would  include  the  investigation  of  multiple 
platform  operations,  as  well  as  the  ability  for  multi-users  to  both  operate  and  co-operate 
within  the  tool. 

Multiple  military  vehicle  types  should  be  considered,  as  well  as  movement  in  three 
dimensions  -  this  could  include  land,  water,  and  air-based  vehicles.  Along  with  terrain¬ 
mapping,  the  application  of  uneven  surfaces,  and  the  consideration  of  pitch,  rolls,  and 
yaw  inherent  in  3D  motion  would  significantly  enhance  the  capabilities  of  the  CONOPS 
Navigator. 

We  intend  to  develop  analytical  metrics  which  incorporate  both  commonly-used  code 
metrics  and  the  complexity  of  interaction  between  the  interfaces  and  Unity  3D  Pro 
scenarios  and  3rd  party  software. 

Another  proposed  avenue  of  investigation  is  to  utilize  Magic  Draw’s  animation 
capability  and  apply  it  to  the  CONOPS  Navigator  architecture  evolution. 
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